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ABSTRACT 

Using four large existing databases on student 
performance in the elementary grades, this study involves a 
task-by-task, year-by-year cross-sectional analysis of six topics in 
computation. Information about what students eventually learn and 
when they learn it is provided by the data analysis; information 
about when skills are introduced and expanded is determined by 
examining school textbooks. This information is seen as being useful 
in setting higher standards for elementary schools, which currently 
concentrate on teaching skills piecemeal over several grades. 
Specifically, topics investigated are addition and subtraction with 
regrouping, multiplication and division facts with numbers 5 through 
9, and multiplication and division beyond basic number facts. Results 
show that the topography of student performance varies across topics 
and across grades within topics. Implications of findings are that 
efforts focusing on learning opportunities in the middle elementary 
grades will be more productive than efforts undertaken at the point 
of entry into secondary schooling. In particular, it is concluded 
that students learn most when topics are taught as part of regular 
instruction, that the margin for school improvement may be more 
promising in certain segments of a subject (i.e., subtraction and 
division), and that a dramatic loss of performance occurs when the 
tirst teaching of a topic occurs late in the school year. (CB) 
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AVOIOIW; PITFALLS IN T« PURSUIT OF HIGHER STAMDARDS FOR ELEICNTARY 
SCHOOLING 

Aaron D. Buchanan 

ABSTRACT 

The pursuit of higher standards for elementary schooling needs to 
take full account of what schools spend their time on now, what students 
in the elementary grades eventually learn and when they learn it. Using 
large existing data bases on student performance in the elementary 
grades, this research involves a task-by-task, year-by-year cross- 
sectional analysis of six topics in computation. Results show that the 
topography of student performance varies across topics and across grades 
within topics. The implications are that focus and timing of learning 
opportunities for all or nearly all btudents in the middle elOTientary 
grades will be more productive than efforts focusing on the point of 
entry into secondary schooling. 

ERIC 



AVOID INC PITFALLS IN T»« PimSUIT OF HIGtCR STANDARDS FOR ELE^NTARY 
SCHOOLING 

Aaron Buchanan 

Although the pursuit of higher standards is currently concentrating 
on secondary schooling, the pursuit will inevitably pick up elementary 
schooling before the quest for educational exceM^nce is over. If the 
same get-tough legislation for ^secondary schools is extended to 
elementary schools, acfeninlstrators and t^.achers will get new marching 
orders to do what they have always been trying to do: raise scores on 
tests of basic skills, the same unfriendly tests of achievement in math 
and reading that have always been a nemesis. This means that, within the 
elementary school ranks, teachers and acfcnlni strators will face the same 
uncertainty they had before the nation began it's pursuit of excellence 
over what tasks students should be required to denKjnstrat e by the end of 
each grade I evel . 

What school officials knw, but what legislators, government 
officials, and education scholars often lose sight of, is this: setting 
standards means one thing In thinking about high school graduates and 
whether or not they*ve had a sufficient number of the right kinds of 
courses to enter the university; it means something quite different in 
looking at a continuous w^ of six grade levels in the elementary 
school— one that has few natural seams for demarcating ^VeaT' standards 
at each grade 1 evel . 

It is possible to set grade-by-grade standards in the elementary 
grades, of course, but the results are fairly arbitrary. Since 
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instruction on almost any of the basic skills that are taught in 
elementary school typically stretches out over t¥K> or three grade levels, 
few of the tasks that often make up standards for the elementary school 
seem like Important benchmarks In the growth, grade by grade, of what 
students are evmtually supposed to know and know how to do. 

Standard setting is especially difficult ii the elementary grades 
because schooling at this level does not deal In tai^lble units, such as 
fixed, year-long cmrses, that are tied closely to topics that are 
covered In classroom instruction. Fixed courses are hard currency in the 
secondary school. There, standard setting and standard raising involve a 
fairly straightforward process for a<^{n I strators and teachers of adding 
real ccMjrses, having definite beginning and ending points, onto existing 
graduation requirements. By contrast, elementary school adninistrators 
and teachers must deal with the slipperier proposition of raising their 
expectations against units of schooling that tend to float across a long 
expanse of time having no sharp demarcations In vrtiat Is taught from grade 
to grade. Here, the goal Is not so imich to complete something as It Is 
to add value to the aggr^ate perfomance of an undifferentiated mass of 
students entering kindergarten or moving through six succe^lng grade 
1 evel s. 

Instead of conpleting courses, the current elementary school 
curriculum is concentrated ot the development of sk i 1 1 s— expans I ve and 
conplicated ones that are taught piecemeal over several grade levels. 
Any piece of a developing skill that Is taught within any single grade 
level seldom has a beginning or ending point that is very striking. 
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Students who successfully learn what is taught will have va1u« added to 
what they were able to do before, but they won't have reached a point 
where they can do much of anything next year except to learn more pieces. 
What students have learned by the end of any single grade level way well 
')e significant in relation to what they Itnew before, but the nature of 
what students have caapleted is not very significant. 

At any point In time, the substance of what elementary stucSents 
should itnow or Icnow how to do is frequented. As elementary schooling 
currently worlcs, nothing is ever really taught once and for all at any 
elementary grade level, much less learned by an entire grade level's 
worth of students. Always, there are pieces to be added by teachers at 
another grade level. Since the end points of instruction at any single 
grade level seldom represent the con^letion of something that, by Itself, 
is very useful, most grade-by-grade standards that are estabi ished for 
the development of a basic sitill in the elementary schooi arc Indefinite. 
The development of a skill is never really coR^lete, at least not with 
the same degree of finality that a course I* coH^leted, because there is 
always some task, usually a more complicated one, that could be added to 
students' repertoires. To make things even more Indefinite, the pieces 
of a developing skill that elementary teachers teach at one grade level 
are usually retaught extensively at the next grade level and are of ten 
reviewed at still another grade level. This recycling has the effect of 
stretching real benchmarks (i.e., things that students need to be able to 
do with reliability and confidence) over several grade levels. 



The conventions that are often used to set standards are not very 
coiT^atible with the conv^tions of schooling in the elementary grades. 
Standard setting tends to conc«itrate on in^at individual students need to 
icncw or i<noM hc^ to do before they iwve to another level of Instruction. 
On the other hand^ the process of schooling in the elementary grades 
d^mds on timely acquI$ItI<Mi of if^rtant skills by a generation of 
studmts. Host standard setters have their thinking grounded In the 
performance of academic tasks that Individual students fmist demonstrate 
before th^ can enter successfully into secondary courseiiork. Too often, 
they have little understanding of how these performances actually "groi/* 
within a g«ieration of stud«its across the elementary grades. As a 
consequence, most standards consist of tasks that individual students 
must perform either Just before they leave elementary school or very near 
the point where these students enter secondary school. (Strange as it 
may seem, the standards for leaving elonentary school are often somewhat 
lower than standards for regular entry into secondary school.) Higher 
standards are established by requiring individual studmts to perform 
more tasks or more con^licated tasks near the point of entry into 
secondary school. Host often, these standards can only be met through 
massive remediation, also near the point of entry into secondary school. 

When standards are set grade by grade for the elementary school, 
they often don't al \gn grade by grade with the bulk of opportunities for 
teaching and learning that are provided In school textbooks. When 
misalignment occurs, It's usually because standards have been set too 
conservatively. The academic tasks that students must perform to meet 



standards at a particular grade level are often stressed one or two grade 
levels earlier in textbooks. The misalignment of standards with learning 
opportunities is often most seriwis when standards become 'Minimal 
competencies" that Individual students must demonstrate before they can 
be promoted to regular, non-remedial instruction at a higher grade level. 
There Is nothing unique about the academic taslcs that constitute minimal 
competencies, but there Is usually something wrong with the pclnts in 
students' grade-by-grade progress where they are required. Because the 
academic tasks involved In minimal coi^etencies often act as gates to 
instrt'ctlon at a higher grade level, the inability of stud^ts to perform 
them successfully carries the stigma of fall'ire, not only for Individual 
students, but, to some extent, for teachers. Students are often required 
to perform the academic tasks Involved In minimal con^etencies well past 
the grade level **«re nwst learning c^portunltles might be expected to 
occur* As a consequence, minimal competencies have come to r^resent a 
"last" chance for Individual students to perform routine academic tasks 
rather than a '•best" chance to learn them. 

CHART I IK OIOUTH IN THE PERFOIWAMCE OF SCfWOLIIIfi 

What Is missing In the current call for higher standards Is a sound 
awareness of students' cismilatlve accon^l Ishments on academic tasks as 
they move through the elementary grades. Without a clear picture of what 
effects the schooling process Is producing now, It is difficult to know 
where the margins for In^rovlng this process are located. Many standards 
are set well past the point where "working harder" on the tasks would be 



most beneflcfat. Other standards are set in place at one grade level 
even th<»igh many prerequisites for these tasics are not now being learned 
by many students. Without these prerequisites in place at eariier grade 
levels, the progression of schooling cannot worit as intended. 

The intelligent pursuit of excellence should mean more for the 
elementary school than establishing standards that amount to little more 
than elaborate but regressive mechanisms for triggerir^ remediation. 
What this pursuit should lead to is adjustments made early enough In the 
schooling sequence so that risks to the regular progress of a generation 
of stud«Yts on routine academic tasks can be reduced. Often, these 
adjustments begin with full exploitation of opportunities that exist in 
that part of school instruction that is, In practice, sustained by school 
textbooks. Full exploitation does not Imply that individual teachers 
must attend slavishly to a sequence of textbook pages. Wiat it means is 
that administrators and teachers should t^e full advantage of what they 
can know about critical points In the mainstream of grade-by-grade 
instruction for a generat Ion of studorts riAtere major opportunities for 
teac>^ing and learning are most likely to occur. 

Full exploitation of learning opportunities means knowing as much as 
possible about the gaps that exist In the grade-by-grade sequence n^ere 
regular learning opportunities are scheduled to occur In school textbooks 
and the points where a critical mass of students, say 751 'o 851, are 
able to demonstrate the accomplishments that these learning opportunities 
are designed to produce. Sometimes, nothing needs to be adjusted because 
the gap between opportunity and accon^lishment has little Impact on other 
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things that students laust learn how to do for the process of schooling to 
move along smoothly through several grade levels. At other times, thest; 
gaps are Important, because new learning opportunities assume that 
certain routine tasks are already In place. These gaps trigger formal 
remediation, which moves large msnbers of students onto separate tracks 
%«here catching up becomes unlikely. 

Making adjustments In the growth performance on routine academic 
tasks require an understanding about two thingst 

(1) How an academic skill Is Instructed task by task and grade by 
grade— when the most rudlawntary tasks are intro^ed; when the 
range of siller tasks Is expanded to Include more complicated 
ones; wh«i the capability to do certain routine tasks becomes a 
"taken-for-granted" part of work on some other task. 

(2) What a large cross section of students knows how to do 
grade-by-grade; what proportion of the aggregate of all students 
at particular grade levels can perform a specific academic task; 
how this performance changes as the momentum of teaching moves 
from a less inclusive task to a more inclusive and more 

compi Icated one. 

Looking at the details o* task -by-task performance by large groups 
of students Is not something the education community has done very imjch, 
especially from a clear perspective of when these tasks ordinarily 
receive concenfated Instruction In regular elementary school programs. 
When It comes to higher standards, the concern has focused on "new" 
Ideas, techniques, and materials to remedy deficiencies In student 
performance at the point where these deficiencies become envious and 
troublesome. 

Part of the trouble has been a lack of interest by most researchers 
in the anatomy of school tettbooks, the primary conduit for most learning 
opportunities that focus on rojtlne academic tasks. Another thing that 
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has h^ered task-by- task analysis of stud»Yt performance has been 
overrel lance by researchers on conventional standardized achievement 
tests as their data source. Unfortunately, the Information that Is most 
accessible from these tests comes In the form of percentiles and 
grade^equlvalwit scores trfilch are ^Inly Intended to show how the 
performance of Individual students compare with the average performance 
of all students nationwide who are at the sane grade level. Percentiles 
and grade equivalent scores reveal nothing about the aggregate 
performance of a generation of students <m particular academic tasks. 
For exw^le, it is possible for Individual studmts (or all students In a 
classroom, or all students at one grade level) to have high grade- 
equlval^t scores w\ a range of tasks ¥A\m\ measured against the 
population of all studmts in the nation at that grade level. At: the 
same time, the priH>ortfon of this population of stud^ts that can perform 
each of the separate academic tasics may be quite 1ok#« In doling with 
percentiles or grade-equivalent scores, the meanings of phrases such as 
'^pursuit of excel lerae** and 'Valsing standards'* get lost In a shifting 
frme of reference— v^at is good or not good Is relative to how well 
everyb<Hly else performs. 

Fortunately, it Is possible to get a better understanding of studmt 
performance on routine academic tasics using other kinds of infonration 
from conventional, standardized achievement tests and from other data 
sources that are currently available* With 'this information, we can do a 
reasonably good Job of generating some tasl(-by-taslc profiles of 
performance by an aggregate of students at different elementary grade 
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levels, in general, the curmilatlve Instructional acconpl ishments of 
schooling can be sketched out by looking first at the proportion of 
students, all at the sasie grade level, who can perform some routine 
academic task. Second, vie can see hcM this performance charges as the 
emphasis In Instruction moves, grade by grade, to Include the task's more 
complicated "relatives." In a very practical sense^ this kind of 
analysis gives us some feeling for how well a generation of stud^ts 
tends to keep up with Instruct Ion— not, as with norm- referenced 
information from conventional achievement tests, how well Individual 
students keep up with everybody else. 

The growth of student performance on a single task might be expected 
to }ock like the single l^rnlng curve shown below. There is a huge 




TIME 



increase in the proportion of students who can do a particular task early 
in Instruction. But as instruction continues, the Increases in the 
pr<^ortion of students mrf^o can perform the task become snaller and 
sma! ier. 
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In practice, the topography of the performance of a generation of 
students Isn't this clean, especially %#hen teaching Is stretched out over 
several related tasks at different grade levels. A lot depends on the 
point In time when one looks at performance of a cross section of 
students during the school year. For exai^le, we should expect seme 
decrease over the summer In the proportion of students who can do a 
particular task. For almost all students there Is a gap of two or three 
months between the end of one grade level, when a task Is introduced, and 
the beginning of the following grade level, before It Is reviewed. 
Second, we would also expect some dips In performance as Instruction 
shifts from simpler tasks to more Inclusive ones. It's also possible we 
might see some eventual decline In performance because of forgetting, but 
It's unlikely because routine acadanic tasks are ones that get a lot of 
use in the learning of other things. 

To the extent that elementary schooling Is successful, we would 
expect to see the curve on a series of closely related tasks to 
eventually level off at a point where more than 80* or even 90* of all 
students at a grade level are successful. On other series of tasks, 
where schooling Is less successful, we would expect to see the 
performance curves flatten out at a much lower level. 

Are there practical limits to how well a generation of students can 
perform routine academic tasks? The obvious answer Is yes. If one looks 
at performance of a group of students on some rcxjtlne academic task, no 
more than 100* of the students can come up with the right answer. But 
seeing 100* who can do anything Is rare, especially for a large group of 
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students. Perforwence levels In the range of 90-95* are likely to be 
about as good as school Ing ever gets. 

More Important than the ultimate limits of the performance curves 
are the methods at our disposal to see how these curves actually behave 
now- For ex«iip1e, researchers now have more than ten years worth of data 
accumulated from the National Assessment of Educational Progress (NAEP) 
showing, in great detail, what students at ages 9* 13, and 17 In a 
national sample are able to do. We can also determine, with reasonable 
accuracy, the approximate time when different school topics are likely to 
be taught, just by looking at when these topics are covered In most 
school textbooks. 

To find out more about what students know and *^en they know It, we 
took a careful look at changes over time In the level of student 
performance on several routine tasks that are part of six very 
fundanental topics In elementary school mathematics. These topics 
Include: 

• Addition with regrmjping (carrying) 

• Subtraction with regrwping (borrowing) 

• "Hard" imiltlpl Icat ion facts 

• "Hard" division facts 

• Multiplication bey«id basic facts 

• Division beyond basic facts 

The Intent was to find points In time where the basic curve showing 
the level of student performance begins to flatten out after a 
substantial amount of formal instruction has taken place. We also wanted 
to see v^at performance curves look like for two series of tasks that are 
part of two different but related topics in mathematics, and we wanted to 
find out how soon after the end of formal instruction that increases in 
levels of performance begin to flatten out. 
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Four existing data bases were used to examine student performance on 

m 

the six computation topics. T«m> are large SWRL data bases that contain 
information on elementary students at eKh of grades 1-6. A third is the 
entire set of results for students at ^es 9 and 13 v^o were part of the 
second mathematics assessownt of the National AssessiMnt of Educational 
Progress (fMEP). The fourth cfaita base consists of statistics on 
individual Items used In establishing norms for students at different 
grade levels on Form U of the Comprehensive Tests of Basic Ski lis (CTBS). 
Use of all four data bases allowed us to looic at the growth of student 
performance on r<MJt1ne coi^utatlon tasks that are basically taught, but 
not necessarily learned, by the aid of grade k and continue to show 
ii^roven^t at later grade levels. Altogether, the four data bases 
Included information tl.at was generated during a period from 1978 thrmigh 
1982. 

tost of the growth in student performance was determined by looking 
at performances the same or similar items givm to students at 
different grade levels in school. The fx^ulatlon rq>r<»mted by each 
data base was a little different, so the information from each data base 
was k^t sq>arate in the analysis to minimize the likelihood of 
concealing differences between apples and oranges. 

The basic procedure was to retrieve selected Items from different 
data bases for the sane grade level end from different grade levels 
within the s^e data base. The objective was to follow across several 
grade levels the growth of student performance on routine tasks that are 
part of the six tq>ics on computation with whole nun^ers. It was ; 
impossible to follow only a single task, such as 2-dIgit addition, over 

I 

several grade levels, because our data didn't Include addition with only ' 
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2-digit nu(rt>ers past about grade 3- it was possible to pick items that 
represoited gradual increases in the rai^e of tasics that students were 
aslced to do. For exaii^ie» we could shift our attention fron 2-digit 
addition to 3-digit addition, as the analysis moved from grade 3 to grade 
h within our data bases. 

Results of the analysis across the six co^utation topics are 
reported in three sect Ions: Addition and Subtraction; "Hard" 
Huitipi ication and Division Facts; and f^i tlpi icatlon and Division 
Algorithms. Discussion on the first topic, addition and subtraction 
(with carrying and borrowing) , is fairly extensive. The purpose Is to 
demonstrate the analytic procedure for finding places in the 
grade-by-grade sequence of Instruction where the performance curves 
flatten out. Results that deal with the remaining topics are then 
sisnnarlzed with briefer discussions. 

Addition and Subtraction (with Carrying and Borrowing) 

Using the PVS, Essential SIcills, and NAEP data bases, we were able 
to construct a profile of growth on routine tasics involved in addition 
and subtraction with carrying or borrcMlng. We begin with 2'-digit 
nuf^ers, about the middle of grade 2, and end with k- or S-digit nt^ers 
in grades 5 and 6 (see Figure 1.1). From the PVS data base, we piciced up 
perfornwnces on addition and subtraction at mid-year in grade 2, which is 
about the earliest point that textbooics begin to provide instruction. By 
the middle of grade 2, a little less than half of the students In the PVS 
data base couid do a routine tasit involved in addition with carrying: 

29 
+16 



m 



ERIC 



15 

Looking at the top curve In Figure 1.1, It Is cl^r that by the ojd of 
grade Z, almost 701 of the students could now do a similar task Involved 
In addition with carrying: 

37 

At the beginning of grade 3> student performance drops back to about 
the point where It had been at the middle of grade 2. By the end of 
grade 3, however, the performance level en a more complicated task 
Involving addition of 3-d1glt numbers with two carries, climbed to almost 
90«. 

239 

Our analysis shifted from 2-dlg!t to 3-dlgIt maimers between the 
beginning and end of grade 3, because there were no suitable tasks 
Involving addition of 2-dlglt numbers In the PVS data base at the end of 
grade 3, However, the shift from two-digit to 3-dIgIt nwnbers Is 
consistent with the shift that textbooks make when they stress addition 
with 3-dlglt nus*ers In grade 3- Addition with 2-dlgIt ntanbers Is 
covered In textbooks for grade 3, but It Is only an Intro&ictlon to more 
conqsl Icated forms of addition Involving nim^ers with three, and sometimes 
four, digits. There is some drop in performance on addition with 3-digit 
nuittoers during the summer between grades 3 and but not nearly as much 
as there was with 2-digit nun^ers between grades 2 and 3> 
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Beyond the «id of grade 3» (Hir analysis doesn't shorn any other 
dr«»atlc Increases In the level of student performance as addition with 
carrying shifts to larger nt^ers. ! terns In the PVS data base shift to 
4-digit nuRd>ers at the end of grade k, c<Nisist«it with a shift in what is 
stressed In textbooks, in the PVS data base, the level of studoit 
performance on Ji-diglt nt^ers hovers near th^Oi level through the 
middle and the end of grade k and then climbs to almost 90% by the middle 
of grade 5* It's fairly safe to assimie that. If we could ccmtlnue to 
look at addition tasks involving only 2-dIgIt and 3-digIt niters through 
grades k and 5, studimt performance might Inch ujymrd beyond the 90S 
level we saw with 3-dIgit addition at the end of grade 3, but It's 
unlikely that they would move very far. Grade 3 Is the last time that 
stud«its get much Intensive Instruction m additI<Ni with carrying. By 
grade k, tentbodks shift their emphasis in conqyutatlon to imilt ipl Ication 
and divisI<Miy so they fMy have only two or three lessons that deal 
directly with addition, and these usually cover subtraction at the same 
time. 

To ext^d our analysis of addition skills beyond grade 5» we lodced 
at performances on cooparable addition tasks In the Essential Skills data 
base. We do not show the actual items in Figure 1.1 , but we do show 
levels of student performance, using hollow triangles, on ^-digit 
addition at the end of grade 5 and on 5-digit addition at the end of 
grade 6. These performance levels appear to be about the same at the end 
of grades 5 and 6 as levels in the PVS data base at the middle of 
grade 5. 
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Student perfon»nce levels In the NAEP data base are generally lower 
than those In our other data bases, especially at grade k, NAEP's second 
assessment of 9-year«o1ds ^Hssn't Include laany I tens on addition with 
carrying, and n«ie of the ones t»"t are In the data base Involve addition 
of two <»-digit mffiid>ers, the Icind of routine tasic we are locking at In the 
PVS data base at grade k. One NAEP iten does involve addition of three 
^-digit numbers. On this tasIc, the perfonmnce level for 9-year-oid 
fourth graders is about 55%$ which is aiiaost 15 percmtage points lower 
than performance levels in the PVS data base on addition with two (rather 
than three) A-diglt numbers. The NAEP data base does have an item 
involving 2-digit addition with carrying. A little more than B0% of the 
9.year-o1ds at grade k answered it correctly. This NAEP level of 
performance on 2-digit addition at grade k is a little lower than the 
levels in PVS for 3-digit addition at the end of grade 3* 

Why are performance levels In NAEP generally lower than performance 
levels In PVS and Essential Skills data bases? It's hard to say. A 
partial answer may reside In the way students are required to do tasks In 
NAEP assessments. NAEP uses free-response items, for which students 
write rather than sel^t the correct ans«#er, for all of Its tasks 
involving basic co^utation with ti^ole ntm^ers. The other t%M> data bases 
Involve multlple-cholce Items, It Is possible that selecting a response 
on a multiple-choice Item Is easier for the kinds of routine computation 
tasks we arc (Ascrvlng. On the other hand, the wrong answers In a 
multiple-choice Itan may actually Increase the likelihood of common 
errors In confutation. The wrong answers In multiple-choice Items often 
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r^resent a very obvious and easybut-wrong way to "attack" the ;.roblem. 
Wh(N) students consider the wroi^-answer choices, it's very Iflcely that 
many of them see a qufcic fix for getting thrmigh with a co^utatlon tasic, 
even though their answer is wrong. Wrong answer choices for co^utation 
tasics usually involve simple interiMdlate steps that are not appropriate, 
but they are easy for students to recognize. Because stud«its can easily 
see how a wrong ansi^r choice could be derived from nis^ers that are 
givm, the answer becoiMS plausible. 

There is am>ther reascm that t^? performance levels may tend to be 
lower than performance levels In the PVS and Essential ^111$ data bases. 
NAEP Items are administer^ under conditions that are very carefully 
controlled to such an extcMit that testlr^ sessions my, in fact, be rigid 
and ur^offifortable for many students. All NAEP Items are administered by 
special ly- trained personnel, not by a student's regular classroom 
teacher. Some directions are read aloud by the person ailnlnl stering the 
assessment; others involve special, tape-n^orded directions. For some 
tasks students are shc*#n the directions; for others they aren't, 
individual students respond to a wide variety of Item types, many 
Involving tasks that are mich more difficult than ones the students are 
regularly asked to do. This cond>{natlon of unfnnlliar tasks and testing 
envirofWient Inherently Introduce more con^l icatlons than students 
ordinarily encounter In testing that Is done In the classroom. 

CTBS performance levels on addition tasks are con^arable to results 
In the PVS and Essential Skills data bases. Figure 1.2 shows performance 
levels <m five addition tasks that cover several different grade levels. 

ERIC 
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The CTBS-derlved perfomnce level on 3-dlgit addition with carrying 
begins at 79% at the end of grade 3- This result Is a little lonrer than 
the BS% level observed In PVS» but It laoves on to almost 90% by the end 
of grade k. The CT8S performance level on addition of three numbers 
having up to four digits begins at about 721 at the md of grade k. This 
is considerably higher than the 601 level observed In NAEP (see Figure 
1.1) on a coBiparable task at the lelddie of grade k, and the performance 
level on this tasic continues to move upward, reaching about 60l by the 
end of grade 6. CTBS Includes tasks that Involve addition of three 
numbers having up to six digits In Its test for students are at the 
end of grade 6, 7* or 8. The level of performance on this task begins at 
almost 801 at the end of grade 6, and moves up to 83t at the end of grade 
8. The B3% level Is about the same as what we see In the NAEP data base 
(see Figure 1.1) on addition of three numbers at the middle of grade 8. 
Thus, It seems that schooling can produce levels of performance on 
addition of whole numbers that eventually reach 80-90) for a population 
of students. 

The growth of stud^it performance In subtraction runs almost 
parallel to performance on addition tasks, but at a some^at lower level. 
The data In Figure 1.1 shows that, for maimers of about the sme size, 
subtraction with borrowing runs any*rfiere from 10 to 30 perc»)tage points 
lower than addition with carrying. The biggest difference occurs at the 
end of grade 3, where about 85) of the students in the PVS data base 
could do a task involving 3-digit addition that required two "carrying" 
steps, but only SS% could do a comparable task involving 3--diglt 
subtraction that required only one "borrowing" step. In part, the 
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difference in difficulty has to do with the fact that subtraction with 
borrcMring involves a sequence of steps that is different from addition 
with carrying, in many ways, the subtraction task may be more 
complicated, even thmigh it r«)uires only one borr<Ming step. For one 
thing, the student must decide whether or not borrowing is necessary 
before beginning to subtract <Nie digit from another. Within the additlcm 
task, the carrying step is taken care of after the initial addition of 
digits has alreacfy begun. But there is another factor that is 
undoubtedly involved. Teachers at grade three will provide Instruction 
on addition of 3-digit nun&ers with carrying to almost all students by 
the Old of grade 3, but many will m>t get to subtr^tion with borrowing, 
especially with students t^o typically require a lot of time and 
patiefKe, before they learn new things. Work on subtraction of 3*>digit 
and 4-digit niffi^ers comes later in grade three textbooks than v^rk on 
conq>ar8bIe forms of addition, and many teachers don't get to it. 

From grade k to grade 6, the level of performance on subtraction 
shows about the sane rate of increase as performance on addition, 
although there Is still a difference of about ten percentage points 
between levels of performance on addition and subtraction. 

By the middle of grade 8, HABP data show performances on addition 
and subtraction of 2-dlglt niters that have come quite close together; 
both of them r^ch a level where about $0% of the grade 8 students are 
successful . 
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Figure 1 »2 Performance Levels on Addition Problems Sfmllar to Itews on CTBS Form U 
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Performance levels derived from CTBS are considerably lower for 
subtraction with borrowing than they are for addition with carrying 
through aVl of the elementary grades. However, the tvto come very close 
to each other by the end of grade 8. As shown In Figure 1.3* a little 
more than 501 of the students at the end of grade 3 can solve a 2-dlgit 
subtraction problem that Involves borrowing. By the end of grade k, 701 
can solve a 3-dlgIt subtraction problem that has one borrowing step, 
while SS% to 651 could solve 4-digit subtraction problems that have one 
or more borrowing steps. The subtraction task showing the lowest level 
of student perfornance at grade k requires three borrowing" steps. It 
is especially tricky, because many students will subtract 8 from 8, 
Instead of 17, In the t«is colunm. By the end of grade 5, the level of 
student performance on this problem moves from 541 to 67*, and It moves 
on up to 7^*% by the end of grade 6. 

Overall, our data on addition and subtraction show the following 

patterns: 



1. The growth of student performance on both addition and 
subtraction takes on about the same zlg-atag profile, especially 
across grades 2, 3, and k >^ere the bulk of regular Instruction 
appears In school textbooks. 

2. A lot of what students learn fn grade 2 Is forgotten by the 
beginning of grade 3, but Is quickly relearned and extended. 

3. nost of the "fn^rovement" In addition and subtraction occurs In 
grade 3> 

4. The level of performance on addition with carrying flattens out 
In grade 3, although the range of performance at the 80-90? 
level is extended In subsequent grades to include addition with 
larger nun^ers. 

5. The level of performance on subtraction does not begin to 
flatten out until near the end of grade k. 
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"Hard" Multiplication and Division Facts 

Using the PVS data base, we were able to track the growth of 
performance on hard multiplication and division facts from grade 3 
thrcMjgh grade 6. No single multiplication or division fact is Included 
In every PVS instrummt. But we were able to follow several closely- 
related tasks through the beginning of grade 6 that require recall of 
multjpl Icatimi and division facts. Textbook lessons on hard 
multiplication and division facts don't begin until grade 3» snd the bulk 
of Instruction Is completed by the middle of grade k. However, 
multiplication and division facts get a lot of Indirect practice when 
students learn and practice algorithms for imiltlplying and dividing 
larger numbers and also when they are learning how to generate equivalent 
forms for common fractions and mlx«j nisabers. 

The results In Figure 2 show that the level of performance on "hard" 
multiplication facts moves from a low at raid-year of grade 3f where a 
little more than 50% of the students In the PVS san^le could find the 
answer to 6 multiplied by 9 . to a high of better than 90% on about the 
same kind of multiplication problem In grade 5. The very high level of 
performance on 9 multlpl led by 5 at mid-year of grade 5 may have 
something to do with the fact that multiplication Involving "fives" has 
more regularity than multiplication Involving "sixes," "sevens," 
"eights," or "nines" and may therefore be easier for students to 
remember. 

On facts that Involve about the 5»ne nu»»*ers, student performance on 

/ 

division starts out at about the same level as performance on 

m 

multiplication. But, after the middle of grade 3, performance on 
division trails performance on multiplication by five to ten percentage 
points all the way through the beginning of grade 5. 
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At grade 3f perforiaance on division In the Essential StcMIs data 
base is very low. Two "hard" division facts have performance levels 
around 551 and the third, which involves division by 5f is at 70!^, about 
the same as the "nines" division fact from the PVS data base. The low 
level of performance on hard division facts in the Essential Stciiis data 
base undoubtedly has something to do with the recommended schedule for 
teaching multiplication and division In the school district where these 
data were obtained. The district's program calls for multiplication and 
division facts to be taught In grade 3 and again In grade k. The 
textbooic series which this district has adopted for regular Instruction 
Includes hard division facts at grade 3, but, Y Ike most other textbook 
series, the lessons come very late In the grade 3 textbook. It is quite 
likely that a large number of teachers don't get to hard division facts 
before the end of grade 3, leaving this instruction for next year's 
teachers to pick up In grade k. 

Coverage of "hard" multiplication and division facts in the NAEP and 
CTBS data bases is spotty. CTBS Form U has no hard imiitlpi ication or 
division facts in any of its tests. MAEP covers hard multiplication 
facts in its assessment of 9-year-olds, but it doesn't cover division 
facts until the assessment of 13-year-olds. 

Within the NAEP data base, the results on two hard nail tipl Ication 
facts at grade k are about 30 percentage points lower than results on 
similar facts from the PVS data base at about the sane point In Che 
school year. Part of this difference may be due to the fact that PVS 
tasks require students to select the best response from among four 
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alternatives, while NAEP tasks require students to write their response 
In a space that's provided. Moreover, multiplication and division facts 
are pres^ted In PVS In much the same way that students would encounter 
thou In a textbook. Students taking MAEP assessments see no written 
problen. Rather, they respond to a phrase such as "five times nine," 
which Is spoken by the MAEP proctor. This situation is quite unlike 
regular multiple-choice assessment Items, which students see a lot. It's 
quite likely that the unusual conditions for testing associated with NAEP 
often lower student performance, especially at grades 3 and k when 
students are first learning how to multiply and divide. By the middle of 
grade 5, performance levels on multiplication and division facts are both 
at about 90* and they continue at about the s^ level Into grade 6. 
Results from the MAEP data base show that the performance level on 
multiplication facts reaches about $0% at the middle of grade 8. 
Performance levels on hard division facts overlap with multiplication a 
little bit, but are slightly lower. 

Overall, the results from the PVS, Essential Skills, and NAEP data 
bases show that the main growth In student performance levels on hard 
multiplication and division facts, about *0 percentage points, occurs 
between the middle of grade 3 and the end of grade k. This Is a time 
span when school textbooks provide most of their lessons that deal with 
hard multiplication and division facts. At the end of grade 3, 
performance levels In these three data bases vary widely, and they 
continue to do so through the middle of grade A. During this period. It 
would be easy to make unreliable assumptions about what students can do. 
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Dlfferef^es In pri^len format or in the kind of response that's required 
m^e a big difference In what individual students are able to 
de^Mistrate. There Is a fair-sized drop In performance levels that takes 
place over the sunaer bet%«eai grades k and 5> but, by the middle of grade 
Sf after mul tipl IcatlcMi and division facts are quite 1 licely to be given a 
revIeM by classroom teachers, performance levels flatten out at a place 
t^ere 85-901! of the students are ^1e to consistently recall 
imjl tipl Icatlon and division facts. 

Hultipl ication and Division Algorithms 

Our analysis of algorltNis for multiplication and division of larger 
nui^ers shoirs the sme upumrd zig-zag pattern In performance that we saw 
with "hard** multiplication and division facts. Beginning midway through 
grade 3» PVS data In Figure 3.1 show that ^xjt 301 of the studmts can 
multiply a 2-dIgIt nund>er by a 1-digIt ntai^er, and that a little less 
than 301 can do a co^arable form of division. By the end of grade 3f 
almost 703; of the students can now do basic miltiplication-wi th-carrying 
and more than SQ% of them can do the sme basic kind of division. 

The second half of grade 3 is many stud«its get their first 
instruction on imjltipl Ication and division of 2-dIgIt and 3~d}glt numbers 
by a 1-digit nimiber, although some teachers may leave this Introductory 
work to be done !n grade 'i. Oft^ tochers choose instead to provide 
more practice on multiplication and division facts. All textbooks 
provide several lessons on multiplication of 2-diglt and 3-diglt nianbers 
by a 1 -dig It nun^er before the end of grade 3, and most of them also 
introduce a co^arable kind of division Involving 1-diglt divisors. All 
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of this work Is repeated In grade kf but at a imjch earlier point fn grade 
h textbooks. Often Mork In the first half of grade k Is actended to 
multiplication and division by tens (e.g., 36 x 20). Before the end of 
grade kf most textbooks move to a more g«iera1 (and nore co^llcated) 
algorithm that Involves multiplication and division by any 2-dIgIt 
nunber . 

Classroom review of the basic imiltlpl Icatlon and division algorithms 
c<NitInues all the way throi^h grade 6, partly because these algorltlms 
are coo^llcated and ti^e a loi^ time to learn. Another reason Is that 
basic algoritlmis for multlplyli^ and dividing t^ole movers are also used 
in mul tipl icaticNi and division with decisis, nrfiich textbooks Introchice 
In grade 5 and 6. Therefore, It is very useful to review miltlpl icatlon 
and division with whole nun^ers just before new work Involving 
multiplication and division with decimals Is begun. 

We followed the growth of multiplication and division algorithms by 
looking at performance levels to see how they were increasing In relation 
to a broadened r^nge of performance that Includai more co^llcat^i 
multiplication and division tasks. It was In^sslble to follow student 
performance on the sli^le forms of multiplication or division by a 
1-dlg!t nim^er all the way through to the end of grade 6. By the middle 
of grade 5» most textbo<4cs have moved (hi to 2-digit imjitipllers and 
divisors. So have almost all of the Items In the PVS data base. In 
order to have Plough data points to show the "form" of Increasing levels 
of student performance all the way through the end of grade 6, we shifted 
our analysis from 1-dlg!t to 2-dlglt imiltipllers and divisors at the end 
of grade 5. 
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By the «Idd1e of grade 5, there U an increase of about 50 points fn 
the percentage of stud^ts Mho can do problems in multiplication and 
division beyond basic con9>utation facts. There is a slight, but 
expected, drop in level of student perfomance on multiplication taslts 
betwm the middle and the end of grade 5 as our itons shift from 1-digit 
to 2-diglt multipliers. By the end of grade 6, hoi#ever, the performance 
level on 2-digit multipliers recovers to about the same level as 
performance with 1-digit multipliers one grade level earlier. Overall, 
the level of student performance on division trails performance on 
multiplication by anywhere from five to AO percentage points. A huge 
difference occurs at mid-year In grade k. Here, more than 70* of the 
students can multiply a 2-diglt nwaber by a 1-dlglt nimd»er, but less than 
AOI can do a similar problem x^ere they have to divide a 3-dlgit number 
by a 1-diglt number (and obtain a 2-digit quotient). In part this 
difference may have something to do with the tasits, themselves; dividing 
352 by 6 (see Figure 3.1) may not be as ca^arablc to multiplying 38 by 7 
as, say, dividing Sk by 6. However, a large part of the difference In 
difficulty between ccB^arably complicated forms of multiplication and 
division at this point of instruction must have a lot to do with the fact 
that textbook lessons on the division always come at a later point than 
lessons on multiplication. As a consequence, many teachers may be 
deferring instruction on division for many students until the next grade 
level. Many teachers feel that students who typically learn mathematics 
at a slower pace are not "ready" to do serious work on anything as 
complicated as long division at the end of grade 3. Some are even 



36 



32 

reluctant to ask students to do mch with division at grade k. It Is 
alnwst certain that the differoice i#e see In studmt performance levels 
at the «id of grade 5 between miltipl Ication and division by 2-digit 
nuad>ers Is largely a result of the dlffer^tce In ei^hasis that teachers 
tend to give th^e two topics; mil tipl Icatlon by 2-diglt maimers almost 
always gets covered by the aid of grade 5» but the same kind of division 
often gets fK>stponed until grade 6. 

The flattoiif^ out In performance mi multiplication that shows up in 
the PVS data base arwnd grade 5 Is verified by Information on co^arable 
Items from the Essential Skills data base. At the end of grade k, two 
pr<4>Iems in the Esswtial Skills data base that Involve multiplication by 
a 1-dIgit nuRd>er have higher performaiKe levels than a similar task, 627 
X 8, in the PVS data base, while three problems of the same type have 
slightly lower performance levels. At the «td of grade 5» the 
performance levels on imiltlpl Icatlon by 2-dIgIt niters In the Essential 
Skills data base are all slightly lower than performance levels on a 
similar PVS task, %73 x 58. By the aid of grade 6, the tiKO data bases 
agree almost completely* 

The levels of performance on Items In the MAEP data base are a lot 
lower In grade k than performances on con^arable Items In PVS or the 
Essential Skills data bases. By the middle of grade 8, multiplication 
and division by l-diglt nim^ers In the NAEP data base has moved up to 
about the same performance level we saw In the PVS and Essential Skills 
data bases at the middle of grade 5, although the division task derived 
from MAEP, 608 divided by 6 , is much less complicated than the ca^arable 
PVS task, ^97 divided by 8 . 
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NAEP tasks that Involve multiplication and division by 2-digit 
nw^ers are much less co^licated than any compar^le tasks in the PVS 
data base. One NAEP task, 323 x 13 t requires no carrying steps In 
finding the partial pro<ktcts, while another task, h6B divided by 36 . 
obviously has a "1" as the first digit in the divisor. A task derived 
fron NAEP, 3052 divided by 28 . Is more cenpllcated than any of the PVS 
tasks shown In Figure 3<1f because It has a **0'* as the second digit In a 
3'-digIt quotient. This task has an NAEP performance level of about $0% 
in contrast to the performance level on a PVS task, 3789 divided by kS , 
which Is more than 7S% at the end of grade 6. 

CTBS-deriv«l levels of studwt performance on multiplication (see 
Figure 3*2) show a steady growth on tasks Involving multiplication by 
1>diglt numbers. This growth begins In CTBS at about 701 at the end of 
grade k and moves to almost 90% at the end of grade 7* (There are no 
Items Involving multiplication by 1-digIt movers In the CTBS tests In 
Form U that are Intended for studmts at grade 3>) The level of 
performarKe on problems Involving milt Ipl Icat Ion by 2-dlgIt ntm4>ers ti^es 
a dr«ratlc Jin^ of more than 30 percmitage points betwem the end of 
grade 5 and the end of grade 6, and it continue upward to about %5% at 
the end of grade 8. Performance levels In the middle grades are some<«^at 
lower in the CTBS data base than either the PVS or Essential Skills data 
bases, but they eventually flatten out In the Interval of about 80l to 
901 In grades 7 and 8, much like performance levels !n the other two data 
bases. 



3a 




Figure 3.2 Performance Levels on Mul t fpUcatlon Problems Sfmllar to Items on 
CTBS Form U 

3i) 



35 



CTBS performance levels for division show some special chalienges to 
schooling in the elementary grades. As Figure 3*3 shows, school 
instruction through grade 8 is only moderately effective for anything but 
the most straight forward division taslcs. Hone of the division tasics that 
we t«ere able to tracic move above the level of 701 to 80|. Performance 
levels on some special division tasics, having one or two zeros in the 
quotient ( klOBk divided by 21 and 6651 divided by 5 ). show almost no 
growth past grade 6. 

IMPLICATIONS FOR NI6MER STMiDMIDS 
The results of this analysis are important to the consideration of 
higher standards for ele^itary schooling. They are ifflfK>rtant to 
teachers, principals, subject matter specialists, and superintendents and 
their assistants who have the broad responsibilities to bring large 
groups of studoits along in the pursuit of excellence. Outside the 
'domain of elementary schooling, these results are ii^rtant for standard 
setters in government, colleges and universities, and even In secondary 
schools, who are inclined to support directives to "bring up test 
scores," often without benefit of icnowing much about the details of t^iat 
is generally being accomplished now In the elementary grades. 

People who set higher standards for elementary schools cannot assume 
that, because students often perform higher on some routine tasks than 
they do on other areas of the curriculum, the teaching and learning of 
routine academic tasics is not a prd>lem. The assumption Is false. As 
these results demonstrate, many routine confutation tasics come on line 
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for all or nearly all students well past the point in schooling where the 
core of instruction is provided, and well past the tifi» when these tasics 
have become a prerequisite to student participation In other, often 
non-routine, areas of a subject matter. 

As far as details are concerned, higher standards for the elementary 
grades should be concerned with several characteristics in the growth of 
student performance as a result of elemwitary schooling. 

First, the growth In performance that occurs whenever topics are 
part of regular instruction for all or n^rly all students is much more 
dramatic than any growth that Is lllcely to occur later, because of 
intensive efforts to fine tune various aspects of ongoing instruction. 
Mastery learning and pass/fall programs In^ lamented one or two grade 
levels after an academic taslc has been covered In regular textbook 
lessons may be important for some Individual students or even some 
classrooms that include homog«)cous groups of students who are mostly not 
doing very well in school. However, these prograns will not do much to 
Improve the performance of elementary schooling for either the present or 
future generation of students. 

Second, the margins for school in^rovement are more promising In 
some segments of the topography of an academic subject than others. Our 
analysis shows that the margins In elementary school mathematics are most 
favorable around Instruction on the subtraction and division algorithms. 
The levels of student performance on both algorithms trail their 
counterparts In addition and multiplication all the way through the 
development years of basic arithmetic with whole nuna>ers. U would be 
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easy to disregard the difference between subtraction and addition and 
between division and t»1 tipl Icatlon, attending only to the <4>vlous fact 
that subtraction and division are more coi^licated processes than their 
computational siblings. Sotie of the discrepancy that we see between 
student performance on division and amI tipl icatlon or subtraction and 
addition obvlmisly do have roots In Intrinsic differences In coR^lexlty 
in these basic arithmetic skills, but not all. We Icnow that subtraction 
and, especially, division come very late In textbooks the first time that 
these tq){cs are either Introduced or significantly ext^ded to larger 
niters. We also know that teachers often tmd to skip chapters near the 
end of their textbooks. 

With subtraction and division, there are g(K>d*slzed margins for 
school improvement especially if teachers are encouraged to cofr^Tete all 
of the Instruction on subtraction and division tq>ics at each grade level 
where these topics occur— no imtter hw late the Instruction comes In 
textbooks. Often, teachers don't go on to subtraction or division 
because they had quite a bit of trouble teaching addition or 
multiplication. Some students are still a little shaky. While 
sensitivity to the iiwnaturity and anxiety of individual students is a 
characteristic of good teaching, the complete avoidance of difficult 
topics because all or nearly all students are not entirely ready 
r^resents questionable pedagogy. As long as teachers are careful , 
students can do significant and productive work on con^llcated topics 
even though they may not be able to do well on a coir^lete range of tasks 
that these topics entail. For students who are likely to experience a 
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lot of anxiety with coniplicated processes, there are ways, short of 
waiting until next year, to carefully control the difficulty of the tasks 
they are asked to atta^t. 

Third, a drmatic "loss of performance* occurs *^en the first 
teaching on a topic occurs late In the school year, and this loss may 
provide a fairly wide margin for school improvement for mtire schools or 
districts. For each of the topics in this analysis, the Instructional 
treatment on particular tasks comes very late In the textbook and, as we 
noted earlier may be skipped by many teachers. Even when Instruction Is 
provided, the anount of practice that students get on a particular task 
is often quite skinny ^ and may come so late In the year that there Is not 
enough time for periodic strengthening, spaced over several weeks, before 
the year Is out. 

These Iii^l Icat ions are not complicated and neither Is the action to 
Implement higher standards for schooling. From one perspective. It Is 
fairly clear that more than 901 of elementary school students learn basic 
multiplication and division facts by the end of grade 5 and that upwards 
of 801 of than eventually learn how to do routine tasks involved In 
addition and multiplication and probably subtraction. These are 
Important accomplishments, but they're qualified. The phrase . . 
eventual ly learn how to do routine tasks . . Is a big qualifier that 
often upsets well-Intended efforts to set higher standards and, more 
important, to realize them. From another perspective, schooling looks a 
little different. By the end of grade 6, almost 20% of all elementary 
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students are unable to dosonstrate the most straightforward tasks in 
addition and multiplication that involve carrying. More than 20% of them 
can't do basic subtraction. Moreover, fairly high levels of performance 
at grades k and 5 in the PVS and Essential Sicllls data bases (where 
problems are not designed to be especially triclcy and students have 
plenty of time) deteriorate badly when the conditions of performance are 
unusual, as they are with W£P assessments, or t4ien speedlness becomes a 
factor, as it does with CT6S. When "performance" moves away from basic 
forms of computation and into taslcs that have special con^l i cat Ions, such 
as "zeros in the minuend" in subtraction or "zeros In the quotient" in 
division, things get worse yet. 

Overall, the results show that, through grade 6, stud^ts can do a 
lot of different routine tasics Involved In computation, but the 
reliability of their performance is fragile. A pedagogical sledgehoamer, 
even though wielded under the banner of "excellence," is not the right 
tool for improving the performance. Bringing all or nearly all students 
to a status i^ere confutation algoritNns are con^letely reliable—subject 
only to careless or random error — has been a historic struggle for 
schools. In the long range. It should at least be considered that some 
other approach now might malce better use of the limited amount of time 
that's available for teachers to t^ch and students to learn. In the 
past, there was no reasonable alternative to learning complete, full- 
blown algorithms for confutation; now, with calculators and 
mlcrocoof uters, there is. 
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For noM, however, human memory-driven computation fs the dominating 
factor in mathematics instruction for the elementary grades. The 
question Is how much to expect. Even if schools can push performance 
levels on computation problems to Z5'90%, there is still a question of 
timing, it is not enough that students ''eventually learn" how to 
compute. Hany learning opportunities that elementary schools provide at 
one grade have strong lir^ages to skills that will come at the next 
grade. Little that is of any consequ^tce is learned in isolation. As it 
is now, opportunities missed are lUely to be opportunities lost for a 
sizable proportion of learners. Timing is critical. 

From any perspective, the process of elementary schooling is 
complicated. Pursued Intel 1 ig^tly, the successful achievement of 
educational excellence In any school subject requires more than the 
efforts of individual teachers teaching small groups of children. A 
simplistic view of what teachers and principals need to do to promote 
excellence 111 serves all parties— students, teachers, a<ftn{nistrators, 
and the public. Pushing teachers to try harder Is not enough. All In 
all, better articulation of learning opportunities across grade levels 
has a lot to do with raising standards and promoting excellence In the 
elementary grades. Intensity Is a critical part of any worthwhile 
effort— but so is focus and timing. 
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APPE»IX A 

KSCRIPTIOi or TOPICS IIRLIIDED IN THE MMLYSIS 



All six topics in the analysis involve coiiq>utation with %Ao1e 

numbers. Each of the topics receives a substantial mmiber of textbook 

lessons, cooipared to other topics that are taught at about the same grad 

levels, and each one becomes eit>edd«l In instruct Imi on soow other topic 

a grade level or so later. 

Topic 18 Addition of 2". 3". and »-dlqft Niaabers (with carrying) 
Addition of 2-dlglt maabers Is Introduced In seme taxtboeks by the 
end of grade 1» but It doesn't require any regrouping of place 
values (carryli^). Addition with carrying begins In the last half 
of grade 2 and Is expanded In grade 3 to Include 3-dlglt and often 
4-dIglt nuabers. This topic Is reviewed in grade k and Is usually 
extended to addition of nun^ers having more than four digits, but 
Instruction usually amounts to no wore than two or three lessons. 
Textbooks for grades 5 and 6 also Include a small aisount of review 
In most textbook series. 

Topic 2i Subtraction of 2*. and 3-diglt Numbers (with borrowing) 
SubtrTCtion of 2-dlglt ni«d>ers with borrowing begins at the end of 
grade 2. It Is extended at grade 3 to Include 3-digIt nim^ers, but 
sometimes the subtraction does not Involve much work with minuends 
that have zeros (302 - 215) until grade k. Nevertheless, the 
instruction on subtraction with reros and subtraction with 'i-dlglt 
nimibers is completed by the end of grade k and Is reviewed briefly, 
along with addition. In grades 5 and 6. 

Topic 3i '*Hard" >ki1tlpl Icat Ion Facts This topic Includes 
multiplication of the numbers 6, 7, 8, and 9 by the nun*ers 5, 6, 7 
8, and 9. instruction on multiplication facts begins in most 
textbooks at grade 2 with easy facts such as 3 x 2 and 3x4. 
Instruction on hard facts begins In grade 3» usually In the second 
half of the textbook, and is repeated in grade k, Oftwi, there Is 
small amount of review provided near the beginning of grades 5 and 
6, but the bulk of formal Instruction Is completed in grade k. 

Topic k: '*Hard" Division Facts These facts parallel the 
multiplication facts identified above (e.g. 5^ : 6 and k2 i 7). 
Instruction on hard division facts begins late In grade 3» usually 
after all of the work on hard imiltipl ication facts is finished, 
instruction is redone in grade 4, usually In the first quarter: of 
the textbook. A small amount of review and practice Is provided 
early in grades 5 and 6 together with practice on imil tip I ication 
facts. 



Topic 5i HuUtpl icatlon by 1«dlqlt and 2-digU Wimfeers Instruction 
on mil tipl icatlon by 1-dlgit numbers often begins late In grade 3 
and extends well into grade k. Nultlpl Icatlon by 2-digit ntn^ers 
usually begins late In grade k and ext^ds Into grade 5> Although 
multiplication by l-dlgit and 2-digit whole numbers Is revlei#ed 
early In grade 6, there are other grade 6 topics, such as 
multiplication by decimals and multiplication by 3-digit nimbers, 
that build directly upon them. 

Topic 6? Division by 1-digit and 2"diqit Numbers Instruction on 
division by 1-diglt numbers also begins near the end of grade 3 in 
many teKtbooks and Is covert thoroughly by all elementary school 
textbooks in grade k. Division by 2-digit numbers Is introduced 
near the «id of grade k, a1th<Migh rany teachers defer instruction 
until the topic is reintroduced in grade 5< 



BESCmPTiaN OF MTA MSE$ EIVLOYED III TIC MMLYSIS 



The iMjor data base used for grades 1-6 Includes Information from 
SWRL*s Proficiency Verification Systae (PVS) collected at the 
beginning, niddle, and «id of the 1978-79 school year. PVS 
Inventories Involve over 200 assessment Item at each of grades 1-6. 
Developamit of the I tans was based on a careful analysis of hoM topics 
are introduced grade-by-grade In mathematics textbooks that schools 
use most. One feature of PVS Inventories Is especially useful for 
analyzing students' performances on similar assessments across two or 
three adjac«it grade levels. Many mathematics problems that are the 
same, or very nearly so, appear In PVS InventoriM for contiguous time 
periods. Just as there Is overly In Instruction on the same topic 
across two or more grade levels, there Is some overlap In ftams across 
levels of PVS Inventories. For eKample, multiplication and division 
facts are part of PVS Inventories used at the middle of the school 
year In grades 3, 4, and 5; they are Included in Inventories for the 
beginning of the school year at grades 4, 5, and 6| and they are part 
of the end-of-year Inventories for grades 3 and 4. During the 1970-79 
school year, over 1,000 studmts at each grade level took each of the 
three PVS Inventories as part of their school's regular testing 
program. Almost all of the students ««ere located in three large 
school districts whose school populations Included a wide range of 
student capabilities. All of the studmts In each of these districts 
were Involved In the PVS program. Using the PVS data base we could 
look at student performances on similar Items within each topic 
beginning at mid-year in either grade two or grade three and, 
depending on the topic, continuing on through the beginning, middle, 
and Old of grade 5 or grade 6. 

Our alternate data base includes the results from three assessments of 
essential skills given In the sprii^ of 1979, 1980, and 1982. The 
assessments were specially written to fit the objectives of a large 
urban school district's instructional program. Each assessment 
consists of approximately 50 mathematics itass at each of grades 1 
through 6. More than 25,000 students at each grade level are 
administered these assessments at the end of the school year, fn most 
instances, the six topics we wanted to follow are assessed about one 
grade level later in the Essential Skills data base than they are in 
PVS. This happens because of the logic that underlies the assessment. 
The Intention of the school district Is to give as much opportunity as 
possible for Instruction on each objective to take place before 
studoits are assessed. As a result, most of the skills are assessed 
one to tM> grade levels behind where they are Introduced and devel<^ed 
In most textbooks. 



4;) 



The NMEP data base comes from the second nationwide AMthe^tlcs 
assMSffient, iirtitch Mas conduct^ in 1978. It r^n^eits a large s^^ie 
of students at ages 9» 13» and 17 fron across the nation. The NAEP 
data base has a unique structure that w^aiwes the utility of the data 
for some kinds of analysis, butXiimlts It for others. One feature 
that ei^nc^ an analysis of student growth on routine academic tasks 
Is the huge variety of tasks that students are asked to do In each 
NAEP assessm^t. Some tasks are very difficult for students at a 
particular age level, while others are very easy. The nice thing 
about this feature Is that <me can alilKost alMiys find an Item in the 
NMP data base to rq>resent so^ toplc\that Is the focus of some 
research on student performance. \ 

A major limitation of the IMEP data base fpr 1978 is that all 
assessments were a<l»inl stored to studmts «^ were all of the same age 
but not at the smne grade level in school. Therefore, data on 
9-year-old stud«its mostly Includes f mirth graders, but it also 
includes a fairly large mmber of third graders who were 9 yrars old 
In January and February of 1978. In acMitlon, the WEP data base for 
9 year olds Include a small mm^er of students «^ were in grades 2, 
5, and 6 at the time of the assessment. Hfi£P, in its published 
reports, has mainly provided results for «Kh of Its three age groups; 
very little Information has been published about results for different 
grade levels within thrae ^e groups. Fortunately, the data t^es 
tirhich WP has made available to researchers have codes that Identify 
the grade level of studmits t^lng the assess^it. Therefore, it 
possible to look at studoit performnce for different grade levels as 
well as dlffermt chrmiological ages. One (k>es have to exercise some 
caution, however. Performance levels of 9'*year-'Old f mirth graders ^ 
include most of the students m^o are supposed to be in fmirth grade, 
but they would not include very bright o-year-olds, who might be in 
the fourth grade. Even more likely, fWt? results for 9-year-olds 
won't include any lO-y^r-old stud«its who are still in fourth grade 
because they are serlmisly behind. 

In our analysis, we looked mlnly at data for fourth graders, made 
up about two-thirds of the NAEP senile of 9-year-olds, and eighth 
graders, krfto made up ^>out two-thirds of the I^P sanple of 
13-year-olds. Ve did not look at data for the NAEP senile of 
17-year-olds, because these studmts have be«i out of elementary 
school too long for N^P results to tell us imich about effects of 
schooling in the elementary grades. 

The CTBS-derlved data base Includes p values, the proportion of 
students answering a test question correctly, cri^tained from a national 
s«^1e of school stud«it5 by CTB/HcGraw-H1 1 1 when various levels of 
Form U of this test were normed In the spring of 1980. The p values 
used In our analysis actually represntted the "proportion of students 
answering correctly irfw attmyt«i to answer a particular item ." This 
kind of p value excludes students who did not try to answer the 
particular item. We used the "atten^ted" p value, because the CT6S 
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tests are carefully timed. Some slow-Morking students mHo might have 
ansti^^ correctly on items that occurred n«ir the end of the test 
never got to then. Obviously, they didn't answer these items 
correctly, but we (ton't know that ail slow nmrtdng stud«its would have 
answered them incorrectly, either. About the best %«e can do is to 
assuine that the "attempted*' p values for items near the end of the 
test may be a little high as indicators of the proportion of all 
students who would have answered correctly if the test had not been 
timed. 

Altogether, our CTBS-derived data base included information from 
approKlmately SO to 90 items on Levels D through F of Form U. At each 
level, the items were from two subtests: Mathematics Coaputatlofi and 
Hathematlcs Concepts and Applications . 
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DESCRimON OF AMALYTICAL PROCEDURES 



Using the PVS data base we moved assessment by assessment from the 
beginning, to the middle, to the end of the school year, and we repeated 
this process across several contiguous grade levels to see how the level 
and the range of student performance Increased together. Along the My, 
we supplemented various PVS-derlved data points with studait performance 
levels on similar itmiis from our Essential Skills data base. Going 
further, we Included data points for coordinating Items on NAEP results 
for 9-year-old fourth graders. In some cases, the Items we show for HAEP 
were Items actually used In the assessment; In others, the l*-«w we show 
only Illustrates an NAEP Item type because the real NAEP Item Is still 
restricted and cannot be pt^llshed. Beyond grade 6, we used Items from 
NAEP date for 13-year-olds In grade 8 to observe levels of performance 
and, to some extent, ranges of performance several grade levels after 
regular Instruction would have (or should have) been completed. 

We used results on CTBS Items to provide some additional information 
on performance levels of students In grades 3 and k and to fill in some 
gaps in grades 6 and 7. Not a lot of Items in CTBS are useful for this 
purpose, because some topics, such as hard multiplication and division 
facts, simply aren't represented very extensively In any of the tests In 
CTBS Form U. In addition, many Items that are Included In the compu- 
tation portions of CTBS represent extremes In item difficulty. For 
example, division of a 3-<*lgIt ni^er by a l-dlglt number, one of the 
early stages In learning about division beyond the basic facts. Is 
represented In CTBS by two kinds of division tasks. One, for exas^le 
3)636, is extrmnely simple because It Involves only the use of basic 
facts and requires no subtraction. The other, for exafiq)le 3)311 » Is very 
tricky because It starts out with a deceptively simple use of basic facts 
but requires students to work with a zero In the quotient. This kind of 
task represents one of the most complicated forms of division by a 
1 -dig It number. For the CTBS analysis, the Items we show are coordinate 
itaRS that have numbers of ^out the same size and Involve computations 
that require about the sme nmbttr of steps to do. 
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